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FOREWORD 


Technological  advancements  have  led  to  increased  speed,  mobility, 
and  destructive  power  of  military  operations.  To  permit  commanders  to 
make  tactical  decisions  consistent  with  rapid  change  and  succession  of 
events,  information  on  military  operations  must  be  processed  and  used 
more  effectively  than  ever  before.  To  meet  this  need,  the  Army  is  de¬ 
veloping  automated  systems  for  receipt,  processing,  storage,  retrieval, 
and  display  of  different  types  and  vast  amounts  of  military  data.  There 
is  a  concomitant  requirement  for  research  to  determine  how  human  abili¬ 
ties  can  be  utilized  to  enable  the  command  information  processing  systems 
to  function  with  maximum  effectiveness. 

The  entire  research  effort  is  responsive  to  requirements  of  RDT&E 
Project  2Q024701A723,  "Human  Performance  in  Military  Systems,"  FY  I969 
Work  Program,  and  to  special  requirements  of  the  Assistant  Chief  of 
Staff  for  Force  Development,  The  U.  S.  Army  Computer  Systems  Command, 
the  Assistant  Chief  of  Staff  for  Intelligence,  and  the  Combat  Develop¬ 
ments  Command.  Research  is  conducted  by  two  BESRL  Work  Units,  Tactical 
Information  Processing  and  Tactical  Operations  Systems. 

To  conduct  back-up  research  in  close  rapport  with  TOS  development, 
BESRL  in  September  I967  established  a  Command  Systems  Field  Branch  within 
USAREUR.  The  BESRL  Branch  in  Germany  conducts  on-the-spot  human  perform¬ 
ance  research  in  support  of  the  development  of  the  Tactical  Operations 
System  and  its  concepts.  At  the  same  time,  the  involvement  of  BESRL' s 
research  scientists  in  TOS  development  and  design  verification  enables 
the  early  identification  of  field-oriented  human  factor  problems  and 
promotes  experimentation  calculated  to  afford  solutions  of  direct 
operational  applicability. 

The  present  publication  provides  a  synopsis  of  BESRL  research  on 
tactical  information  processing  and  on  tactical  operations  systems,  in¬ 
cluding  principal  findings  and  their  application  to  operational  problems. 
Studies  completed  prior  to  the  beginning  of  FY  I967  were  summarized  in 
BESRL  Technical  Research  Report  1145,  "Human  factors  research  in  comnand 


HUMAN  FACTORS  RESEARCH  IN  COMMAND  INFORMATION  PROCESSING  SYSTEMS— SUMMARY 
OF  RECENT  STUDIES 


BRIEF 


Requirement 


To  provide  an  overview  of  BESRL's  human  factors  research  program  in 
command  information  processing  systems,  with  emphasis  on  accomplishments 
since  March  I966.  Objectives  in  brief: 


Tactical  Information  Processing  (TIP).  To  assist  commanders  and 
their  staffs  in  Information  assimilation  and  decision  making  and  to  de¬ 
velop  techniques  for  the  efficient  processing  and  use  of  information  by 
operational  personnel  in  tactical  situations. 


Tactical  Operations  Systems  (TOS) .  To  maximize  the  effectiveness 
of  command  information  processing  systems  through  the  most  efficient  use 
of  human  abilities. 


BESRL's  manned  systems  research  in  this  area  is  directed  toward  the 
enhancement  of  human  performance  and  facilitation  of  man-machine  inter¬ 
action  in  relation  to  total  system  effectiveness.  It  involves  experimen< 
tation  with  various  configurations  of  system  components,  considering  in¬ 
teractions  and  tradeoffs. 


The  end  products — imnedlate  or  ultimate— are  scientific  findings  on 
human  capabilities  and  performance  under  varying  conditions  within  the 
system.  The  findings  have  implications  for  systems  design,  development, 
and  operational  use.  The  present  report- -the  result  of  a  review  of  re¬ 
cent  research  activities — presents  such  end  products  in  abbreviated  form 
providing  sufficient  information  on  a  given  study  to  show  the  basis  for 
the  findings  reported. 


The  implications  stated  below  must  be  regarded  as  tentative,  since 
development  of  the  Tactical  Operations  System  (TOS)  continues  and  modifi' 
cations  are  still  being  made.  Some  of  the  statements  derive  from  strong 
indications  noted  in  initial  studies.  Continuing  research  will  probe 
more  deeply  the  reasons  for  the  results  obtained. 


Implications  of  Recent  Studies: 

1.  Implication:  The  cathode  ray  tube  (CRT)  should  be  used  for  TOS 
output  as  well  as  input. 


Supporting  research:  For  message  composition,  electronic  devices 
(CRT  with  typewriter  keyboard)  were  found  to  be  superior  to  electro¬ 
mechanical  devices  (teletypewriter).  Advantages  of  the  CRT  held  for  out¬ 
put  procedures  as  well  as  input. 

2.  Implication:  The  present  TOS  transform  process  should  be  changed 
rather  than  merely  aided.  Now,  the  action  officer  selects  a  format,  fills 
it  out,  and  gives  it  to  the  UIOD  operator  who  copies  the  message  on  the 
CRT  screen.  The  action  officer  may  be  able  to  fill  in  the  format  directly 
on  the  CRT  screen. 

Supporting  research :  In  a  field  study,  incorrect  formats  were 
selected  for  22$  of  incoming  messages.  A  specially  developed  device  to 
structure  selection  did  not  improve  performance. 

3.  Implication:  Much  information  now  displayed  in  graphic  and 
symbolic  form  requiring  manual  preparation  should  be  presented  in  alpha¬ 
numeric  form  on  computer- generated  displays.  Proper  use  of  TOTES  could 
reduce  the  time  spent  by  staff  officers  in  preparing  summary  reports. 

Supporting  research :  In  an  experiment  comparing  alpha-numeric 
presentation  (TOTES)  with  graphic  (map  symbols),  amount  of  information 
assimilated,  accuracy  of  decision,  and  degree  of  confidence  in  the  report 
were  not  affected  by  mode  of  presentation.  Results  held  under  time  re¬ 
striction  as  well  as  with  unlimited  time  allowance. 

4.  Implication:  The  present  method  of  rating  spot  reports  should  be 
simplified.  Research  in  the  area  points  up  the  potential  of  subjective 
probabilities  as  a  useful  substitute. 

Supporting  research :  The  present  schema,  requiring  both  a  6- 
point  accuracy  rating  (multidimensional)  and  a  6-point  reliability  rating 
(unidimensional)  was  found  to  be  too  time-consuming  to  be  profitable. 

Even  with  job-aids  to  flow  chart  the  process  of  arriving  at  an  evaluation, 
the  G2  did  not  have  time  to  use  the  rating  schema.  Neither  speed  nor 
accuracy  improved.  The  potential  usefulness  of  subjective  estimates  of 
the  probability  that  a  report  is  correct  was  indicated  in  preliminary 
studies  of  agreement  among  officers  in  rank  ordering  messages  according 
to  perceived  utility.  The  usefulness  of  direct  estimates  of  probability 
is  being  pursued  by  BESRL's  Field  Branch. 

5.  Implication:  The  development  of  an  overall  system  criterion  for 
TOS  must  await  the  determination  of  how  well  the  system  meets  the  specific 
needs  of  different  users. 

Supporting  research:  Preliminary  analysis  of  critical  informa¬ 
tion  requirements  and  varying  and  unique  needs  of  users  as  well  as  the 
fact  that  many  TOS  subsystems  are  still  in  formative  stage  led  to  the 
stated  conclusion. 


6.  Implication:  Attitude  toward  automation  on  the  part  of  TOS  users 
will  have  a  significant  effect  on  the  ultimate  acceptance  of  TOS. 

Supporting  research :  BESRL's  Field  Branch  is  conducting  surveys 
in  the  Seventh  Army  to  assess  initial  attitudes  of  users  toward  TOS  and 
change  in  attitude  after  experience  with  TOS  equipment.  Results  will  aid 
a  program  to  promote  understanding  and  acceptance  of  the  computerized  in¬ 
formation  processing  system. 

7.  Implication:  A  data  base  structure  may  be  shared  by  at  least  the 
three  highest  echelons  of  command,  and  by  armored  and  Infantry  units. 

Supporting  research:  These  elements  were  in  agreement  as  to  the 
types  of  information  commonly  needed  in  army  field  exercies. 

8.  Implication:  Computer  aids  to  tactical  decision  making  may  yield 
substantial  payoff  in  combat  situations  about  which  the  data  are  typically 
conflicting  and  of  low  reliability.  Information  which  is  highly  reliable 
may  be  evaluated  as  fast  and  as  accurately  by  man  as  by  machine. 

Supporting  research :  Subjects  were  found  to  recognize  complex 
patterns  of  activity  when  they  occurred  frequently  ( in  80$  of  the  informa¬ 
tion  displays)  but  not  when  occurrence  was  relatively  infrequent. 


HUMAN  FACTORS  RESEARCH  IN  COMMAND  INFORMATION  PROCESSING  SYSTEMS— SUMMARY 
OF  RECENT  STUDIES 


v  The  Command  Systems  program  of  the  U.  S.  Army  Behavioral  Science 
Research  Laboratory  is  directed  toward  solving  problems  associated  with 
information  processing  and  decision  making  by  commanders  and  their  staffs 
Research  focuses  on  one  or  more  aspects  of  events  which  occur  from  the 
time  a  cotmnander  receives  his  mission  until  he  completes  it.  This  chain 
of  events  is  the  system  within  which  command  decisions  are  formulated 
and  executed— a  tactical  operations  system  within  which  tactical  infor¬ 
mation  processing  is  performed. 


The  Command  Systems  program  is  divided  into  two  work  units.  Re¬ 
search  efforts  which  focus  on  such  information  processing  aspects  as  the 
rapid  and  accurate  input  and  assimilation  of  information  are  conducted 
by  the  Tactical  Information  Processing  (TIP)  work  unit.  Efforts  which 
emphasize  aspects  of  the  total  system  such  as  allocation  of  functions 
to  man  and  equipment  and  system  information  requirements  fall  within  the 
Tactical  Operations  System  (TOS)  work  unit.  The  present  report  traces 
research  progress  from  the  beginning  of  FY  I967  to  the  present.  During 
this  period  23  projects  were  completed  or  are  now  in  progress.  x 


LABORATORY  AND  FIELD  APPROACHES 


Human  information  processing  and  decision  making  may  be  observed  on 
an  abstract  level  in  the  laboratory  or  as  operational  procedures  in  the 
field.  To  provide  a  research  capability  responsive  to  requirements  for 
varying  degrees  of  operational  simulation,  the  Command  Systems  program 
has  established  both  laboratory  and  field  approaches. 


The  laboratory  efforts  are  conducted  within  an  environment  desig¬ 
nated  as  SIMTOS  ( simulated  tactical  operations  system) .  Within  this 
environment,  experiments  are  based  on  simulations  of  tactical  operations 
systems.  System  equipment  is  similar  to  that  now  used  by  the  7th  Army 
TOS  Development  Group  in  Germany.  The  major  devices  are  cathode  ray 
tubes  and  teletypewr iters  linked  to  a  32k  computer  with  disk  and  tape 
storage  capability  consisting  of  three  disk  and  two  tape  drives.  Com¬ 
puter-driven  random  access  slide  projectors,  maps,  and  acetate  overlays 
for  recording  permanent  information  complement  the  SIMTOS. 


To  provide  direct  field  contact  for  operational  applications  of  lab¬ 
oratory  findings  and  early  identification  of  field-oriented  human  factor 
problems  which  should  be  investigated  in  the  laboratory,  the  Command 
Systems  program  maintains  a  field  branch  in  Heidelberg,  Germany,  resi¬ 
dence  of  the  7th  Army  TOS.  Here  it  is  possible  to  observe  at  first  hand 
the  problems  associated  with  the  performance  of  personnel  in  an  automated 
information  system  and  the  evaluation  and  fielding  of  the  system. 


The  maintenance  of  both  laboratory  and  field  facilities  provides  not 
only  a  wide  and  flexible  research  base  but  also  a  means  of  cross-validating 
research  findings.  For  example,  there  may  be  a  question  whether  a  labora¬ 
tory  finding  will  hold  up  in  the  field;  or  a  field  observation  may  need 
to  be  subjected  to  rigorous  laboratory  test. 


REVIEW  OF  BESRL  RESEARCH- -TACTICAL  INFORMATION  PROCESSING 
TECHNIQUES  FOR  RAPID  AND  ACCURATE  INFORMATION  UPDATING 

Screening.  An  automated  information  system  such  as  TOS  receives 
vast  amounts  of  information  from  many  and  varied  sources.  The  informa¬ 
tion  varies  widely  in  content,  form,  and  degree  of  completeness.  Further, 
the  information  often  affects  several  different  staff  groups.  The  raw 
data  require  a  great  deal  of  handling  and  processing  by  man  and  equipment. 
Incoming  messages  must  first  be  screened  for  system  relevance.  Irrelevant 
information  must  not  be  allowed  to  enter  the  system.  The  problem  of 
screening  has  received  little  attention  to  date.  Exercise  observations 
do  not  yield  information  about  screening.  The  scenarios  used  in  command 
post  and  field  training  exercises  employ  only  relevant  messages.  Ambig¬ 
uous  data,  purposefully  deceitful  data,  and  data  without  historical  con¬ 
tinuity  are  rarely  included. 

Transform.  The  process  of  transforming  information  for  input  re¬ 
quires  taking  the  typically  unconstrained  free-English  text  that  charac¬ 
terizes  a  manual  system  message  and  converting  it  to  equivalent  symbolic 
notation,  entered  in  specific  format  fields  (particular  alpha-numeric 
character  locations)  for  the  particular  class  of  information  in  the  com¬ 
puter's  data  base  that  is  to  be  acted  upon.  All  information  contained 
in  the  TOS  data  base  must  be  entered  and  retrieved  via  formats.  If  the 
appropriate  format  is  not  selected,  the  information  is  typically  rejected 
by  the  system,  thereby  introducing  delays  in  updating  the  data  base.  In 
some  instances,  however,  the  information  can  be  processed  through  the 
system  using  an  inappropriate  format.  In  that  case,  the  information  is, 
to  all  intents  and  purposes,  irretrievable;  it  will  not  be  where  it 
should  be  in  the  data  base. 

Research  on  the  transform  operation  in  TOS  has  been  carried  out  by 
the  field  branch  in  Germany  (1).  A  job-aid  was  developed  in  an  attempt 
to  improve  the  accuracy  of  format  selection  by  action  officers  who  must 
decide  which  format  is  appropriate  for  each  incoming  message.  Fourteen 
subjects  processed  47  messages.  The  main  concern  was  the  method  of 
format  selection:  half  the  subjects  used  the  job-aid  while  the  other 
half  used  a  "menu"  type  listing  of  available  formats.  The  job-aid  de¬ 
veloped  was  ineffective  in  improving  the  accuracy  of  format  selection. 

The  more  important  finding,  however,  was  that  an  incorrect  format  was 
selected  22$  of  the  time.  Thus,  the  transformation  operation  represents 
a  definite  bottleneck  in  the  system;  during  periods  of  heavy  message 
traffic,  information  may  wait  in  queue  for  a  considerable  length  of  time 
before  it  is  entered  into  the  system.  Research  is  continuing  to  develop 
effective  aids  which  hopefully  will  yield  error  rates  considerably  below 


I  np»'l  .  Once  tlu>  information  is  recorded  in  the  appropriate  forma  l  , 
it  is  entered  into  the  system.  A  variety  of  input  devices  exist.  In 
response  to  a  request  by  the  Automatic  Data  Field  Systems  Command,  the 
merits  of  electronic  message  composition  devices  ,  such  as  a  cathode  ray 
tube  with  typewriter  keyboard)  and  electro-mechanical  i  tel etypewr i ter ; 
devices  were  compared  (2).  The  advantages  and  potential  of  an  electronic 
message  composition  device  were  found  to  outweigh  those  of  a  teletype¬ 
writer.  Findings  supported  the  procedure  in  the  present  TOS  in  Germany, 
in  which  all  information  is  entered  into  the  system  via  electronic  rather 
than  a  teletypewriter  device. 

INFORMATION  ASSIMILATION  AND  TRANSFER 

At  present,  the  principal  computer-driven  equipment  employed  in  TOS 
is  the  cathode  ray  tube  with  typewriter  keyboard  ( input)  and  the  teletype¬ 
writer  output).  These  devices  transmit  mainly  alpha-numeric  information. 
The  question  arises  as  to  what  extent  information  assimilation  would  be 
degraded  by  the  translation  of  map  symbols  and  other  information  to  alpha¬ 
numeric  equivalents.  If  the  degradation  is  negligible,  then  much  of  the 
information  now  displayed  symbolically  on  maps  could  be  stored  alpha- 
numerically  in  the  computer.  If  the  degradation  is  severe,  either  the 
process  of  generating  map  symbols  must  remain  manual  or  the  Army  must 
wait  for  computer  state-of-the-art  to  develop  a  capability  of  adequate 
reliability  that  can  be  operated  at  an  acceptable  cost. 

A  recent  in-house  experiment  dealt  with  decision  making  using  up¬ 
dated  graphic  vs  alpha-numeric  information  (  ■’)).  Subjects  were  required 
to  decide  in  which  of  three  sectors  the  enemy  was  building  up  for  an 
attack.  Information  was  displayed  via  both  map  symbols  and  alpha-numeric 
TOTES.  Neither  decision  accuracy,  amount  of  information  assimilated,  nor 
degree  of  confidence  was  affected  by  the  method  of  presentation  ( graphic 
vs  alpha-numeric). 

A  second  study  entitled  "Time  Stress  and  Information  Format  In  a 
Decision  Making  Task"  (4)  was  similar  to  the  above  experiment  except  that 
1)  a  time  restriction  of  20  seconds  per  slide  was  imposed  'the  above  study 
imposed  no  restriction),  and  2)  subjects  alternated  between  graphic  and 
alpha-numeric  problems.  Results  were  consistent  with  findings  of  the 
earlier  study  which  indicated  no  difference  in  decision  performance  be¬ 
tween  alpna-numeric  and  graphic  presentation  of  data.  These  results  are 
encouraging  since  they  suggest  that  much  of  the  information  now  being 
posted  manually  on  maps  in  symbolic  form  may  be  stored  in  the  computer, 
making  possible  automatic  updating  of  enemy  and  friendly  unit  identifica¬ 
tions  and  locations. 

This  conclusion  was  supported  in  a  study  conducted  by  the  field 
branch  (5).  This  study  dealt  with  implications  of  the  Command  Systems 
in-house  research  for  displays  in  tactical  information  processing.  One 
of  the  conclusions  is  that  the  potential  of  TOTE-type  displays  for  facil¬ 
itating  information  presentation  and  assimilation  appears  substantial, 


especially  of  TOTES  which  tan  be  generated  by  an  automated  data  processing 
system.  Proper  use  of  such  TOTES  could  reduce  reliance  on  situation  maps 
and  save  the  time  of  staff  officers  who  prepare  summary  reports.  Subse¬ 
quent  research  will  attempt  to  determine  more  precisely  what  information 
now  in  map  and  symbol  format  may  be  converted  to  TOTES  and  summary  tables 
with  no  loss  in  performance. 

EFFECTIVE  AIDS  IN  THE  DECISION  PROCESS 

Research  in  this  area  is  aimed  at  discovering  how  decision  making 
ability  may  be  enhanced  or  aided  by  factors  or  conditions  within  the  de¬ 
cision  environment.  Two  such  conditions,  amount  of  information  and 
knowledge  of  results  or  feedback,  were  investigated  in  a  study  entitled 
"Timeliness  and  Accuracy  in  a  Sequential  Decision  Making  Task"  ( 6). 

While  accuracy  varied  directly  with  amount  of  information,  feedback  had 
little  effect  on  either  accuracy  or  confidence  in  performance;  perform¬ 
ance  accuracy  was  57$  under  the  no  feedback  condition  and  65$  under  the 
feedback  condition.  It  was  suggested  that  the  failure  of  feedback  to 
produce  large  changes  in  accuracy  of  decision  may  be  attributable  to  the 
nature  of  the  experimental  task.  Subjects  who  received  feedback  may  not 
have  been  able  to  relate  this  information  to  any  systematic  procedure 
for  arriving  at  a  decision. 

There  was  also  indication  tlmt  lack  of  confidence  in  his  ability  to 
make  accurate  decisions  may  cause  the  decision  maker  to  delay  taking 
action  even  when  he  could  make  an  accurate  decision  on  the  basis  of  the 
information  available.  This  finding  suggests  that,  along  with  techniques 
to  enhance  the  quality  of  decisions,  techniques  are  needed  to  enhance 
confidence  in  those  decisions. 

A  second  study  entitled  "Confidence  and  Posterior  Probabilities  in 
an  Interpretative  Decision  Task  (7)"  is  in  progress.  In  the  interpreta¬ 
tive  task,  a  sequence  of  nine  slides  depicted  a  build-up  of  enemy  forces. 
Each  slide  represented  an  independent  sighting  of  equipment.  The  sub¬ 
ject's  task  was  to  estimate  probabilities  concerning  the  type  of  unit 
building  up  and  to  commit  himself  to  action  when  he  was  reasonably  con¬ 
vinced  that  he  had  enough  information  for  a  decision.  Variables  of 
interest  included  a  computer  aid  and  two  work  methods.  Time  to  decision 
(number  of  slides),  accuracy  of  decision,  and  confidence  estimates 
(subjective  probability)  were  the  measures  to  be  analyzed.  Preliminary 
findings  indicate  that:  1)  a  simple  computer  aid  (furnishing  cumulative 
data  about  enemy  equipment  sighted)  relieved  the  subjects  of  time- 
consuming  clerical  chores,  reduced  errors,  and  thereby  resulted  in 
improved  performance;  and  2)  a  work  technique  which  requires  that  subjects 
consider  all  relevant  information  in  a  complex  task  improved  performance 
over  individual  "common  sense"  approaches. 

The  field  branch  in  Germany  has  directed  much  of  its  attention  to  de¬ 
veloping  aids  for  the  evaluation  of  G2  spot  reports.  The  standard  NATO 
schema  requires  that  each  spot  report  contain  two  ratings,  an  accuracy 
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rating  on  a  six-point  complex  multidimensional  scale  and  a  reliability 
rating  on  a  six-point  unidimensional  scale.  At  first,  research  effort 
was  concentrated  on  job-aids  (8).  Two  job-aids  were  developed,  one 
following  the  Field  Manual  text,  the  other  incorporating  suggestions 
from  the  faculty  at  the  G2  Intelligence  School  located  in  Oberammergau, 
Germany.  The  job-aids  were  variants  of  a  decision  process  flow  chart 
consisting  of  a  series  of  yes-no  questions  leading  to  the  appropriate 
evaluations.  When  the  results  of  job-aided  groups  were  compared  with 
results  of  control  groups  using  no  job-aids,  no  differences  were  found 
either  in  speed  or  accuracy  of  evaluations.  This  finding,  coupled  with 
conversation  with  school  faculty  and  data  from  student  questionnaires, 
led  to  the  following  conclusion:  A  job-aid  is  not  the  solution  to  the 
G2's  evaluation  problems;  he  does  not  have  time  for  such  a  rating  schema 
even  when  the  necessary  decision  process  is  flow  charted  for  him. 

However,  the  above  conclusion  does  not  imply  that  an  underlying 
evaluation  process  does  not  operate.  A  second  study  (9)  demonstrated  a 
definice  utility  of  spot  reports.  Intelligence  school  students  were 
asked  to  select  from  among  16  spot  reports  the  most  important  set  of 
four  messages,  followed  by  the  second  and  third  sets  of  four.  The  re¬ 
sults  yielded  significant  agreement  among  students.  If  students  do 
rank  order  spot  reports  according  to  a  perceived  utility  but  do  not 
have  time  to  use  a  job-aid,  perhaps  the  evaluation  process  could  be 
improved  by  changing  the  rating  schema  itself.  At  present,  the  field 
branch  in  Germany  is  conducting  research  to  test  the  hypothesis  that 
many  G2  personnel  implicitly  assign  a  likelihood  to  the  information  in 
a  spot  report  and  then  convert  this  to  one  of  the  six  categories  of 
rating  accuracy  (10).  If  this  hypothesis  is  correct,  much  information 
is  lost  in  the  process  of  translating  the  subjective  likelihood  evalua¬ 
tion  into  the  objective  rating  schema.  Research  is  now  in  progress  to 
determine  the  extent  to  which  G2  personnel  can  assign  subjective  prob¬ 
abilities  to  the  reliability  of  source  and  accuracy  of  a  report  and 
whether  these  probabilities  correlate  with  the  six-point  scale  currently 
used . 


REVIEW  OF  BESRL  RESEARCH- -TACTICAL  OPERATIONS  SYSTEMS 
INDIVIDUAL  AND  SYSTEM  PERFORMANCE  MEASURES 

BESRL' s  simulated  tactical  operations  system  (SIMTOS)  as  described 
earlier  represents  only  a  "first-cut"  simulation.  Data  from  the  first 
human  performance  experiment  in  which  21  field  grade  officers  (4  colonels 
17  lieutenant  colonels)  served  as  subjects  are  now  being  analyzed.  The 
results  of  these  analyses  will  influence  the  direction  of  subsequent 
SIMTOS  research. 

A  report  describing  a  projected  five-year  program  for  the  SIMTOS 
facility  has  been  drafted  (11).  The  report  consists  of  two  main  sections 
the  first  describes  the  present  simulation  capabilities  of  SIMTOS;  the 
second  is  a  discussion  of  future  SIMTOS  designed  to  meet  additional  re¬ 
quirements  as  simulation  fidelity  is  increased  to  include  many  highly 
complex  capabilities  such  as  real-time  updating  and  staff  interaction. 
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In  the  first  SIMTOS  experiment  (12),  recently  completed,  each  subject 
was  informed  that  he  was  to  take  the  part  of  a  G3  of  a  mechanized  infantry 
division  on  alert  status  in  assembly  areas  near  the  town  of  Kulmbach, 
Germany.  The  subject  was  given  the  Corps  Ops  Order  (less  Annexes)  to 
provide  him  with  a  general  understanding  of  the  defensive  situation,  the 
capabilities  of  his  comnand,  and  the  nature  of  the  enemy  threat. 

The  scenario  was  designed  to  provide  three  measures  of  decision 
making  behavior.  In  the  first  portion  of  the  scenario,  the  subject  is 
requested  to  draw  avenues  of  approach  and  key  terrain  on  the  situation 
map  and  report  his  recomnended  form  of  defense.  In  the  second  portion, 
he  is  to  develop  a  course  of  action  within  the  division  sector  and  allo¬ 
cate  maneuver  elements  to  the  echelons  of  defense--general  outpost,  for¬ 
ward  defense  area  forces,  and  reserve  forces--and  to  specify  the  type  of 
resistance  by  each  echelon  of  defense — delay,  screen,  defend.  In  the 
third  portion  of  the  scenario,  the  subject  is  to  develop  the  graphic 
portion  of  his  defense  order  to  include  location  of 

General  outpost 
Combat  outpost 

Brigade  boundaries  and  coordinating  points 
Battalion  positions  of  forward  defense  area  forces 
Reserve  forces 

Visualized  allowable  penetrations 
Division-directed  blocking  positions 
Task  organization 
Mission  to  subordinate  units 

The  subject  was  to  obtain  information  by  phoning  his  request  to  the 
operator  of  a  user  input/output  device  (a  role  taken  by  the  experimenter) 
who  located  the  information  in  the  data  base  and  sent  it  to  the  subject 
via  CRT,  typewriter,  or  slide  projector.  The  principal  data  were  the 
subject's  answers,  his  information  requests,  and  time  of  his  requests. 

The  results  of  the  experiment  will  provide  preliminary  guidelines  for 
establishment  of  an  overall  system  performance  criterion  and  identifica¬ 
tions  of  critical  information  requirements. 

The  establishment  of  an  overall  system  criterion  is  not  expected  to 
be  an  easy  task.  A  report  in  progress  (13)  concerned  with  user  oriented 
criteria  for  the  TOS  points  out  that  the  system  is  in  an  evolutionary 
phase  and  that  criteria  change  with  each  new  stage  of  development.  For 
example,  while  the  present  version  of  the  TOS  in  Germany  deals  only  with 
typewriter  message  output,  second  generation  versions  may  place  greater 
emphasis  on  the  cathode  ray  tube  as  an  output  device.  Further,  a  com¬ 
puter-generated  situation  display  is  likely  to  be  used  in  the  next  gen¬ 
eration  TOS.  The  report  succinctly  states  the  difficulty  of  determining 
system  criteria  under  varying  display  devices: 

The  practical  consequence  of  this 
evolutionary  development  of  means 
of  displaying  TOS  -  generated  in¬ 
formation  is  that  the  rate  and 
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nature  of  information  assimilation 
by  TOS  users  can  be  expected  to  vary 
with  the  nature  of  the  display  being 
used.  Further,  exactly  how  informa¬ 
tion  is  assimilated  under  various 
display  conditions  and  the  relative 
efficiency  and  effectiveness  of 
different  techniques  remains  unknown 
at  this  time. 

A  second  problem  encountered  in  outlining  a  general  approach  to  the 
user  criterion  problem  is  that  a  system  such  as  the  TOS  has  several  users. 
The  extent  to  which  men  interact  directly  with  the  system  delineates  the 
several  types  of  user.  The  type  of  user  who  interacts  most  closely  with 
the  TOS  is  the  UIOD  operator.  He  is  there  because  the  TOS  is  there.  His 
job  did  not  exist  in  the  manual  system.  His  interaction  is  largely  a 
mechanical  one:  He  types  and  sends  messages  already  composed  by  the 
staff  action  officer,  and  he  receives  messages  sent  by  the  computer. 
Performance  variables  relevant  to  this  type  of  equipment  operator  are 
already  covered  in  existing  human  factors  literature. 

A  second  type  of  user,  whose  duties  existed  before  the  advent  of 
the  TOS, is  the  staff  action  officer.  Before  the  TOS,  this  individual 
performed  his  information  dissemination  role  without  use  of  ADP.  Now, 
he  serves  as  the  first  flexible  interplay  point  between  the  TOS  and  the 
military  staff.  His  job,  the  mechanics  of  which  vary  substantially  from 
staff  element  to  staff  element  (for  example,  from  G2  to  G3),  has  two 
major  functions.  He  converts  staff  information  needs  to  a  form  that  the 
TOS  can  understand  (via  TOS  message  formats).  He  also  has  the  task  of 
inserting  data  to  the  TOS  and  extracting  information  from  it  to  fulfill 
staff  needs.  What  he  does  has  direct  impact  on  the  TOS  and  upon  other 
staff  officers  using  the  TOS. 

A  third  type  of  user  falls  into  a  broader  category,  which  includes 
senior  staff  officers  and  conmanders  who  utilize  TOS  generated  informa¬ 
tion  in  military  operations.  These  officers  are  the  ultimate  TOS  users 
who  work  as  skilled  professionals  performing  executive  operations.  A 
good  deal  of  their  activity  is  more  art  than  straight-forward  science 
and  is  difficult  to  assess  quantitatively. 

The  difficulty  of  deriving  a  criterion  satisfactory  to  many  users 
is  apparent.  How  develop  an  overall  criterion  which  is  sensitive  not 
only  to  how  the  system  is  being  used  but  also  to  which  staff  element  is 
using  it?  Any  general  measurement  approach  must  be  geared  to  take 
account  of  the  unique  situations  which  TOS  users  will  present  to  the 
system. 

One  of  the  very  real  problems  which  the  field  branch  brings  to  the 
attention  of  the  in-house  research  staff  deals  with  the  impact  of  auto¬ 
mation  on  Seventh  Army  users.  In  the  research-development-production- 
distribution  gamut,  those  in  the  research  environment  may  lose  sight  of 
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the  fact  that  users  of  new  systems  may  not  be  as  intimately  acquainted 
with  them  as  are  the  research  scientists  who  devise  them.  User  reaction 
to  a  new  product  must  be  favorable--a  successful  research  program  may  be 
nullified  by  a  user's  refusal  to  accept  innovations. 

in  a  report  now  being  prepared  by  the  field  branch  (14),  it  is 
pointed  out  that  when  the  TOS  equipment  is  delivered  to  the  users  and 
incorporated  into  the  conduct  of  a  field  exercise,  it  will  provide  a 
rare  opportunity  to  measure  the  initial  impact  of  automation  on  staff 
functions  at  three  echelons  of  operation--army,  corps,  division.  It 
will  also  provide  opportunity  to  measure  a  factor  critical  in  predicting 
a  system's  success  or  failure,  namely,  user  acceptability. 

The  field  branch  has  begun  a  series  of  attitude  and  user  observa¬ 
tion  surveys  in  the  form  of  questionnaires  administered  to  members  of 
the  Seventh  Army.  The  initial  questionnaire  was  administered  before  the 
user  was  exposed  to  TOS.  These  data  mainly  provide  base-line  attitude 
measures  concerned  with  the  user's  preconceived  notions  regarding  the 
potential  advantages--or,  in  some  cases,  disadvantages--of  automation  in 
a  tactical  setting.  Upon  completion  of  a  TOS  orientation/training  course, 
the  user  was  again  surveyed.  The  assumption  here  was  that  the  user  should 
have,  upon  completion  of  the  course,  an  understanding  of  the  system  which 
should  be  reflected  in  more  definitive  notions  about  the  relative  advan¬ 
tages,  or  disadvantages,  of  operating  in  an  automated  environment. 

Since  the  initial  increment  of  TOS  includes  only  enough  equipment 
to  supply  one  corps  and  two  divisions,  the  remaining  corps  and  two  divi¬ 
sions  surveyed  were  not  involved  in  the  TOS  orientation/training  program. 
Having  a  non-user  group  at  two  essentially  equated  echelons  provides  the 
opportunity  to  assess  changes  in  attitude  as  a  function  of  new  experiences 
encountered  or  occurring  simply  with  the  passage  of  time.  This  non-user 
control  group  will  be  resurveyed  to  obtain  these  data.  The  user  group 
will  also  be  resurveyed  at  approximately  the  same  time,  and  after  members 
have  had  rather  extensive  on-the-job  training  experience  with  the  system, 
to  obtain  measures  of  changes  in  general  attitude  as  well  as  specific  measures 
of  their  assessment  of  specific  TOS  features  and  implementation  procedures. 

From  the  overall  results  of  the  survey,  it  will  be  possible  to  de¬ 
termine  what  may  be  expected  in  terms  of  initial  user  attitude  toward 
automation  and  to  detect  any  change  in  attitude  following  familiarity 
with  automated  equipment.  In  addition,  results  from  the  survey  will 
delineate  shortcomings  of  the  present  TOS  in  Germany.  Knowledge  of  de¬ 
ficiencies  will  provide  an  empirical  foundation  for  the  assignment  of 
retrofit  priorities  and  provide  a  basis  for  making  design  decisions  con¬ 
cerned  with  future  United  States  Army  command  and  control  systems  such 
as  TOS-75. 

Another  report  in  progress  concerns  a  flow  chart  model  of  the  TOS 
system  (15)*  This  model  will  depict  critical  human  functions  such  as 
screening  messages  for  TOS  relevancy  and  selecting  appropriate  formats. 

The  model  will  also  delineate  those  functions  which  will  be  automated. 
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such  as  standing  requests  for  information.  The  model  will  serve  two  main 
purposes:  to  focus  useful  small-scale  subsystem  experiments  in  relation 

to  the  overall  system,  and  to  describe  the  impact  of  parameters  at  each 
node  (subsystem  operation)  of  the  total  system.  In  the  latter  function, 
the  model  should  bo  particularly  useful  for  evaluations  of  TOS  effective¬ 
ness  . 


COMMANDER'S  INFORMATION  REQUIREMENTS 

The  introduction  of  automated  equipment  may  increase  the  flow  of 
incoming  information  to  such  an  extent  as  to  overwhelm  the  commander  and 
his  staff.  To  preclude  such  an  eventuality,  research  has  been  conducted 
and  is  in  progress  to  determine  the  categories  of  information  which  are 
most  critical  to  the  commander  and  his  staff,  to  determine  how  specific 
information  needs  vary  according  to  the  TOS  user,  and  to  derive  logically 
the  problems  which  could  result  when  the  evaluation  process  occurs  within 
the  TOS.  Answers  to  these  questions  will  not  only  facilitate  information 
flow  within  the  automated  system  but.  also  insure  a  viable  manual  back-up 
system  as  well. 

In  a  report,  "Human  factors  experimentation  within  a  tactical  opera¬ 
tions  system  (TOS)"  (16),  which  contains  a  brief  description  of  the 
Seventh  Army  TOS  now  under  development  and  the  activities  of  the  BESRL 
field  branch  in  Germany,  the  early  endeavors  in  establishing  initial 
information  areas  are  discussed.  As  a  result  of  the  work  of  the  system 
analysts  of  the  TOS  Development  Group  (a  group  made  up  of  military  and 
civilian  personnel  from  the  then  Automatic  Data  Field  Systems  Command, 

Fort  Belvoir)  and  the  efforts  of  a  sizable  number  of  civilian  contractors, 
five  critical  functional  information  areas  were  given  first  priority  for 
automation. 

However,  nothing  was  done  to  insure  that  field  commanders  would  not 
be  buried  in  data  in  the  five  areas,  albeit  all  relevant  data.^  Thus,  a 


^  An  individual  largely  responsible  for  the  fact  that  the  development  of 
the  TOS  is  taking  place  in  an  operational  setting  (Seventh  Army  in  Europe) 
is  the  former  Deputy  Director,  Defense  Research  and  Engineering,  the 
Honorable  Eugene  G.  Fubini.  In  his  opening  address  to  the  Second  Congress 
on  the  Information  System  Sciences,  Dr.  Fubini  warned  against  this  very 
situation.  In  his  words:  "And  so,  facing  the  users  today  is  another 
problem.  It  is  the  problem  of  trying  to  tell  you  technologists  what  it 
is  we  want  you  to  do  .  .  .  We  have  some  partial  answers.  One  answer,  a 
negative  answer,  is  that  we  don't  want  you  to  be  squirrels  when  you  build 
equipment  for  us.  We  don’t  want  you  to  store  things  just  because  you 
think  that  maybe  we’ll  need  them.  To  store  and  retrieve  all  available 
data  is  just  wrong.  Collect  only  what  you  can  use.  I  know  it  is  easy  to 
say  but  difficult  to  implement.  But  still,  if  somebody  comes  in  and  tells 
me,  'You  see,  I  can  collect  everything,  retrieve  everything,  and  display 
anything,'  I  say,  'I  don't  want  it'."  (Information  System  Sciences:  Pro¬ 
ceedings  of  the  Second  Congress.  (J.  Spiegel  and  D.  Walker,  eds.) 
Washington,  D.  C.  Spartan  Books,  Inc.  1065,  page  1.) 


further  it  el  ine.it  ton  is  necessary  to  establish  message  priorities  within 
these  areas.  One  of  the  first  tasks  undertaken  by  the  BESRL  field  branch, 
therefore,  was  to  identify  and  validate  the  types  of  information  being 
handled  hv  the  various  tactical  operations  centers.  An  in-urogress  study, 
information  requirements  for  field  army  users"  (ly),  was  designed  to  de¬ 
termine  how  specific  types  of  information  needed  by  TOS  users  vary  as  a 
function  of  1)  the  particular  staff  element  [G2,  G3,  Chemical,  Biological, 
Radiological  element  (CBRE),  or  Fire  Support  Coordination  element  (FSCE)|; 
')  the  echelon  of  command  (army,  corps,  division);  and  J)  the  type  of  unit 
involved  ,  Armor  or  Infantry). 

Eighty-six  experienced  staff  officers  in  the  United  States  Seventh 
Army,  including  the  principal  staff  officers  in  each  section,  responded 
to  a  questionnaire  containing  61  types  of  information  commonly  appearing 
in  staff  exercise  journals.  Of  the  items,  34  dealt  with  enemy  informa¬ 
tion,  lfi  with  friendly  unit  information,  and  11  with  CBRE  information 
about  friendly  and  enemy  units.  Staff  officers  could  check  up  to  30  of 
the  items  on  the  basis  of  their  importance  during  field  exercises.  While 
all  analyses  of  the  data  have  not  yet  been  completed,  preliminary  results 
indicate  that: 

1.  The  staff  element  in  which  an  officer  serves  is  far  and  away  the 
most  important  determinant  of  what  he  feels  is  of  importance. 

2.  Echelon  of  command  has  no  effect  upon  the  importance  attached 
to  different  classes  of  information. 

3.  By  and  large,  whether  users  are  in  an  armored  or  infantry  divi¬ 
sion  has  either  no  or  little  practical  impact  upon  what  users  judge  to 
be  important  items  of  information. 

4.  On  the  whole,  G2  and  GJ  personnel  checked  all  JO  items  denoting 
kinds  of  information  important  in  field  exercises,  indicating  that  staff 
officers  prefer  a  higher  volume  of  traffic  in  the  G2  and  GJ  areas  during 
exercises  than  in  FSCE,  which  checked  roughly  20  items,  or  CBRE,  which 
checked  about  lj. 

In  an  unpublished  sunmary  of  the  Seventh  Army  manual  system  base-line 
data,  the  field  branch  recently  discovered  material  bearing  on  the  above 
findings.  The  summary  data  showed  that  of  the  messages  coming  into  a  TOC 
approximately  48$  was  intended  for  the  G2,  37$  for  the  G3,  and  lj$  for 
the  FSCE /CBRE.  These  traffic  proportions  were  the  same  at  all  echelons-- 
that  is,  echelon  of  command  had  no  effect.  Thus,  if  need  should  arise  to 
assign  a  priority  of  the  effort  to  be  expended  toward  examining  particular 
classes  of  information,  G2  information  should  rank  high. 

Further  data  relevant  to  the  last  point  appeared  in  a  recently  pub¬ 
lished  field  branch  study,  "Certitude  judgments  in  an  operational  environ¬ 
ment,"  (18).  All  messages  filed  by  two  divisions  of  one  corps  during 
seven  days  of  a  field  exercise  were  examined.  It  was  found  that  spot 
reports  accounted  for  7°$  of  all  G2  messages.  Since  G2  messages  consti¬ 
tute  about  half  of  all  TOC  traffic,  and  since  the  preponderance  of  the 
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G2's  messages  are  spot  reports,  clearly  this  is  a  class  of  information 
to  which  considerable  attention  should  be  paid. 

It  merits  comment  that  division  is  the  lowest  echelon  now  serviced 
by  TOS.  Consequently,  all  information  coming  into  a  division  TOC  has 
been  only  manually  processed  and  thus  requires  further  handling  before 
it  can  be  input  to  TOS.  Since  the  bulk  of  this  traffic  will  be  made  up 
of  G2  spot  reports,  which  nearly  always  originate  from  troops  in  contact 
and  long  range  patrols,  the  application  of  source  data  collection  tech¬ 
niques  to  the  handling  of  G2  spot  reports  has  the  potential  of  providing 
the  biggest  single  improvement  to  date  in  the  conduct  of  tactical  opera¬ 
tions. 

The  fact  that  G2  information  is  so  important  was  one  factor  which 
prompted  a  thorough  investigation  of  the  G2  spot  reports.  The  study  was 
also  prompted  by  concern  over  the  extent  to  which  reliability  and  accuracy 
ratings  were  being  omitted  from  the  G2  spot  reports.  TOS  requires  the 
presence  of  both  ratings  or  it  will  reject  the  message.  It  was  found 
that  if  standard  forms  were  not  provided  and  a  spot  report  had  to  be 
filed  on  plain  paper,  it  did  not  contain  the  necessary  reliability  and 
accuracy  ratings.  It  was  also  found  that  three-fourths  of  all  G2  spot 
reports  were  given  the  same  rating  (high  reliability,  high  accuracy). 

It  was  in  response  to  these  findings  that  the  G2  spot  report  job- 
aid,  discussed  earlier,  was  developed.  It  may  be  recalled  that  the  job- 
aid  was  found  to  be  ineffective.  On  the  basis  of  the  present  and  earlier 
discussion  concerning  G2  spot  reports,  it  may  be  concluded  that  the 
crucial  importance  of  G2  information,  the  failure  to  assign  ratings  when 
standard  forms  were  unavailable,  and  the  tendency  to  give  the  same  ratings 
to  all  incoming  messages  suggest  reevaluation  of  the  entire  rating  schema. 

The  nature  of  the  mission  also  may  dictate  the  critical  information 
needs.  When  the  data  from  the  first  SIMTOS  experiment  are  analyzed,  it 
will  be  interesting  to  compare  the  types  of  information  requested  most 
frequently  by  the  field  grade  officer  subjects  with  the  types  of  infor¬ 
mation  indicated  as  most  crucial  by  the  staff  officers  who  responded  to 
the  survey  on  information  requirements  for  field  army  users.  The  survey 
was  conducted  without  regard  to  any  specific  mission,  whereas  in  the 
SIMTOS  study  the  subjects  were  G5's-  Each  subject  was  instructed  in  a 
mission  statement  that  he  was  to  defend- -prevent  enemy  from  penetrating 
west  of  "line  red."  Thus,  while  the  survey  and  excercise  data  suggest 
that  G2  data  are  most  critical,  the  SIMTOS  study  may  well  find  that 
friendly  unit  information  (G3)  was  requested  most  frequently.  Such  a 
result  would  further  underscore  the  difficulty  of  developing  an  overall 
system  criterion,  since  the  measure  would  have  to  take  into  account  not 
only  the  particular  user  but  also  the  particular  mission. 

To  support  research  concerned  with  the  flow  of  tactical  information 
across  the  sequence  of  actions  in  making  and  executing  a  military  deci¬ 
sion,  a  tactical  decision  information  network  was  developed  (19).  Deci¬ 
sion  points  were  defined,  decision  makers  specified,  and  information  re¬ 
quirements  delineated.  These  data  were  then  organized  in  the  form  of 
function  flow  block  diagrams  and  associated  data  input  sheets  which 
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collectively  constitute  the  network.  The  network  provides  a  framework 
within  which  comprehensive  tactical  decision  situations  (offensive  or 
defensive)  in  a  nuclear  or  non-nuclear  combat  environment  can  be  de¬ 
veloped  for  use  in  research  in  decision  making. 


man-computer  decision  making 

Thus  far,  the  computer  and  its  associated  equipment  have  been  dis¬ 
cussed  as  an  information  storage  and  retrieval  system.  Certain  of  the 
critical  information  requirements  of  the  commander  and  the  lines  of 
communication  from  the  field  to  him  have  been  established.  What  remains 
unknown  are  many  of  the  processes  which  intervene  between  the  comnander's 
assimilation  of  the  information  and  his  issuing  of  orders.  One  of  the 
processes  is  the  aggregation  of  a  considerable  amount  of  information  of 
varying  degree  of  accuracy  and  reliability  as  a  basis  for  an  overall 
evaluation--of  an  enemy  threat,  for  example.  Models  exist  which  will 
optimally  combine  or  aggregate  many  items  of  information  provided  man 
can  first  quantify  the  uncertainty  of  each  item.  Thus,  it  is  important 
to  know  man's  ability  to  evaluate  items  of  information.  Several  questions 
have  been  asked  concerning  this  ability.  What  is  the  relationship  between 
a  man's  performance  and  his  confidence  in  the  accuracy  of  his  performance? 
The  answer  to  this  question  would  provide  evidence  toward  determining  if 
there  are  consistent  tendencies  to  overestimate  or  underestimate  the  un¬ 
certainty  associated  with  individual  items  of  information.  A  second 
question  may  be  asked:  Is  any  one  way  of  expressing  uncertainty,  such 
as  betting  odds,  any  better  than  another,  such  as  subjective  probabilities 
The  computer  is  indifferent  to  the  form  of  the  uncertainty  expression;  the 
form  is  easily  convertible  to  any  metric  by  the  computer.  If  one  response 
form  is  better  than  any  other  (yields  probabilities  which  more  closely 
approximate  truth),  the  use  of  that  form  will  reflect  man's  best  ability 
to  quantify  the  uncertainty  of  information,  and  the  computer  aggregation 
of  these  uncertainties  will  provide  the  best  evaluation  possible.  A  third 
question  is:  What  is  man's  ability  to  aggregate  many  items  of  information 
into  an  overall  evaluation  and  does  this  ability  vary  with  the  degree  of 
uncertainty  of  the  information?  If  there  are  certain  situations  in  which 
man  performs  optimally  in  aggregating  many  items  of  information,  then  the 
cost  of  constructing  elaborate  and  expensive  computer  programs  to  evaluate 
information  for  these  situations  may  not  be  warranted.  Another  question 
is:  How  well  can  man  recognize  patterns  of  activity,  and  what  is  the  need 
for  computer  aids  which  will  provide  automated  pattern  recognition?  Again 
if  there  are  situations  in  which  man  is  perceptive  and  flexible  in  his 
ability  to  discern  patterns  within  a  sequence  of  events,  it  may  not  be 
necessary  to  automate  this  aspect  of  the  decision  process  for  these  situa¬ 
tions  . 

With  regard  to  the  first  question  concerning  the  extent  to  which 
feelings  of  confidence  or  certitude  appropriately  reflect  a  person's  in¬ 
formation  assimilation  performance,  two  in-house  experiments  were  con¬ 
ducted:  "Relation  of  Certitude  Judgments  to  Characteristics  of  Updated 
Symbolic  Information"  (20)  and  "Subjective  Probability  and  Decision 


Behavior  in  a  Perceptual  Task"  (21).  As  is  often  the  case  in  research, 
these  studies  uncovered  many  problems  in  attempting  to  relate  confidence 
and  accuracy.  The  first  is  one  of  measurement.  In  the  first  study,  it 
was  found  that,  because  of  the  different  metrics,  it  was  not  feasible  to 
determine  quantitatively  the  degree  of  conformity  or  whether  the  con¬ 
fidence  judgments  were  predominately  overestimates  or  underestimates  of 
accuracy.  In  neither  study  were  subjects  provided  with  feedback  (knowl¬ 
edge  of  results)  during  the  experiment.  It  might  be  argued  that  in  the 
absence  of  feedback,  a  consistent  difference  between  confidence  and 
accuracy  reflects  a  constant  error  which  would  disappear  if  the  subject 
could  see  that  he  was  overestimating  or  underestimating  by  the  same 
amount  each  time.  Another  problem  encountered  in  attempting  to  relate 
confidence  and  accuracy  is  how  accuracy  is  defined.  The  second  study 
tested  subjects'  ability  to  identify  the  number  of  flag  symbols  on  a 
display  which  was  presented  for  a  very  brief  period.  They  also  indicated 
the  degree  of  confidence  in  the  accuracy  of  their  identifications.  The 
number  of  symbols  varied  from  4  to  24.  It  was  concluded  that  subjects 
typically  overestimated  their  performance.  This  conclusion  was  reached 
on  the  basis  of  comparing  confidence  estimates  with  the  percent  of  the 
time  subjects  identified  the  exact  number  of  symbols.  No  credit  was 
given  for  being  close.  However,  fitting  linear  models  relating  numer¬ 
ousness  to  estimated  number  of  symbols  revealed  that  the  model  with  unit 
slope  and  intercept  equal  to  zero,  using  the  actual  number  to  predict 
the  estimates,  accounted  for  99  percent  of  the  variance  of  the  estimates 
around  zero.  If  credit  were  given  for  being  close,  it  might  be  concluded 
that  subjects  typically  underestimated  their  performance.  From  the  vary¬ 
ing  effects  of  differences  in  metric,  feedback,  and  criterion  definition, 
it  must  be  concluded  that  the  relationship  between  confidence  and  accuracy 
will  not  be  easily  determined. 

The  question  of  response  form  and  ability  to  aggregate  information 
has  been  investigated  in  an  in-house  study,  "Probability  Estimates  in 
Tactical  Decision  Making"  (22).  Form  of  response  was  compared  by  having 
subjects  make  estimates  both  in  the  form  of  odds  and  as  subjective  prob¬ 
abilities.  Ability  to  aggregate  was  observed  by  comparing  human  esti¬ 
mates  of  posterior  probabilities  with  computer- aggregated  posterior  prob¬ 
abilities.  The  study  also  used  a  wide  range  of  probability  levels, 
making  it  possible  to  determine  performance  at  varying  levels  of  infor¬ 
mation  uncertainty.  Data  are  now  being  analyzed. 

The  final  question  dealt  with  human  ability  to  perceive  patterns 
within  a  sequence  of  events.  A  recently  conducted  in-house  study, 
"Perception  of  Military  Event  Patterns  in  a  Two-Choice  Prediction  Task" 

(23) ,  was  designed  to  observe  subjects'  ability  to  recognize  second-order 
patterns;  i.e.,  patterns  in  "hich  the  next  enemy  activity,  attack  or  rest, 
is  predictable  on  the  basis  £  his  last  two  activities.  Subjects  exhibited 
a  high  degree  of  pattern  recognition  when  the  patterns  were  occurring  80# 
of  the  time  but  very  little  pattern  recognition  when  patterns  were  occur¬ 
ring  only  68%  of  the  time.  Evidently,  when  there  is  considerable  pattern 
interference  as  at  the  68#  level,  pattern  recognition  by  the  computer  would 
be  a  useful  aid  to  the  decision  maker.  However,  for  patterns  occurring  at 


the  8o$-level  or  above,  computer  assistance  may  not  be  necessary,  since 
men  appear  to  be  capable  of  recognizing  second-order  patterns  at  these 
levels . 


CONCLUSIONS  AND  IMPLICATIONS 

This  review  has  been  an  attempt  to  describe  briefly  and  to  inte¬ 
grate  the  last  three  years  of  research  within  the  laboratory  and  field 
branch  of  the  Comnand  Systems  program.  BESRL  Technical  Research  Report 
1145,  "Human  Factors  Research  in  Comnand  Information  Processing  Systems" 
(24)  describes  research  conducted  prior  to  this  period. 

Based  on  the  research  described  in  the  present  report,  a  number  of 
conclusions  and  implications  may  be  drawn.  The  following  statements  were 
judged  to  be  the  most  important  operational  implications  to  be  derived 
from  the  research  program.  Statements  are  made  without  qualification  in 
order  to  provide  a  clear  indication  of  the  implications.  They  should 
not  be  interpreted  as  final  in  any  respect.  Army  Tactical  Operations 
Systems  are  only  in  the  incubation  period.  Therefore,  conclusions 
reached  on  the  basis  of  TOS  and  TOS-related  research  must  be  regarded 
as  tentative.  With  these  precautions  in  mind,  the  following  recomnenda- 
tions  are  offered: 

1.  CRT's  should  be  used  as  output  devices.  The  TOS  currently  uses 
the  CRT  primarily  as  an  input  device.  Many  of  the  advantages  which  favor 
the  CRT  over  the  typewriter  as  an  input  device  hold  true  for  the  CRT  as 
an  output  device  as  well. 

2 .  Efforts  should  be  directed  toward  changing  rather  than  aiding 
the  present  TOS  transform  process.  The  present  method  of  having  an 
action  officer  select  a  format,  fill  it  out,  and  hand  it  to  a  UIOD 
operator  to  be  recopied  onto  the  CRT  screen  is  redundant.  The  possi¬ 
bility  of  having  the  action  officer  call  up  a  format  and  fill  it  in 
directly  on  the  CRT  screen  should  be  explored . 

5 .  Much  graphic  and  symbolic  information  currently  presented  on 
manually  prepared  displays  should  be  presented  in  alpha-numeric  form  on 
computer  generated  displays.  Recent  laboratory  comparisons  of  assimila¬ 
tion  of  graphic  vs  alpha-numeric  information  have  revealed  that  subjects 
perform  as  well  with  alpha-numeric  as  with  symbolic  displays  of  informa¬ 
tion. 

4.  Efforts  should  be  directed  toward  changing  rather  than  aiding 
the  present  two-part  spot  report  rating  schema.  The  present  rating 
schema  is  complicated  and  observers  do  not  have  time  to  use  a  job  aid. 
They  may  be  better  able  to  express  their  evaluations  by  using  subjective 
probabilities  rather  than  the  present  scales. 
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' ‘ •  It  is  too  early  to  determine  a  single  overall  system  criterion; 
a  multiple-criterion  approach  would  he  bolter  for  present  experimental  j  . 
The  TOS  is  in  a  state  of  evolution;  there  Is  no  assurance  that  the  present 
man-machine  configuration  will  not  change.  The  TOS-(”^  system  may  he 
radically  different  from  the  present  TOS  in  dermany.  Also,  the  fact 
that  there  are  many  different  TOS  users,  each  with  his  own  unique  needs, 
suggests  that  criteria  should  be  established  for  individual  sets  of  users 
and  that  an  overall  criterion  should  not  be  developed  until  the  system 
has  achieved  more  maturity. 

d .  Attitude  toward  automation  on  the  part  of  TOS  users  will  have  a 
significant  effect  on  ultimate  acceptance  of  TOS.  A  program  for  coping 
with  possible  initial  biases  or  prejudices  toward  TOS  due  mainly  to  a 
lack  of  understanding  of  and  therefore  a  distrust  of  such  "gadgets"  as 
computerized  information  processing  systems  should  be  developed. 

7.  Echelon  of  command  (Army,  Corps,  Division),  type  of  unit  armored 
vs  infantry),  and  assignment  (Corps  A  vs  Corps  B)  will  most  likely  be 
able  to  share  the  same  data  base  structure.  These  groups  were  in  agree¬ 
ment  as  to  the  types  of  information  commonly  needed  in  army  field  exercises. 

>.  Potential  computer  aids  to  decision  processes  ( such  as  threat 
evaluation)  may  be  most  advantageous  for  situations  in  which  the  data  are 
of  low  diagnostic  value.  Subjects  do  not  recognize  complex  patterns 
except  when  they  occur  frequently.  On  the  other  hand,  information  which 
is  highly  reliable  (such  as  friendly  unit  information)  may  be  evaluated 
as  well  by  man  as  by  machine.  For  these  situations,  the  payoff  may  not 
be  sufficient  to  warrant  the  cost  of  developing  programs  to  evaluate  such 
information.  For  other  situations,  such  as  the  evaluation  of  intelligence 
data  which  are  typically  conflicting  and  of  low  reliability,  a  much  larger 
payoff  may  be  derived  from  the  development  of  computer  aids  to  assist  the 
tactical  decision  making  of  the  commander. 
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